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O 
Commonwealth 


Vision 
Ensure each and every student is empowered and equipped to pursue a successful future. 
Mission 


To partner with districts, schools, families, students, business and industry, and communities 


to provide leadership and support to ensure success for each and every student. 


About 


The Kentucky Department of Education is a service agency of the Commonwealth 
of Kentucky, and part of the Education and Workforce Development Cabinet. The 
department provides resources and guidance to Kentucky’s public schools and districts 
as they implement the state’s P-12 education requirements. The department also serves 


as the state liaison for federal education requirements and funding opportunities. 


INTRODUCTION 


This report is a continuation of our work examining the progress being made by districts to close the current achievement 

gap in Kentucky.' In the previous study, we found a number of promising examples of districts outperforming their predicted 
achievement gaps in 3rd grade mathematics. This study extends that work by using our growth measure to investigate the context 
in which districts are closing the achievement gap. Districts’ achievement gap issues vary significantly. For example, the size 

of districts’ achievement gaps as well as the pace at which their gaps close (or widen) have large implications for their ability to 
address the issue. A more nuanced understanding of these differences allows districts to more effectively tackle their achievement 


gap challenges and assists the Kentucky Department of Education (KDE) in better serving districts’ individual needs. 


This study aims to parse out these differences and identify areas of progress and concern. We combine our growth measure 
with districts’ actual gap change in school years 2016-2017 to 2017-2018 to better understand the variation in achievement 
gap progress in the state. We focus our analysis on third grade students’ reading achievement. However, additional analyses not 
reported in this report show similar results.” Additionally, we limit our analysis to districts that had an achievement gap between 


high and low-income students in the 2016-2017 school year (n = 168). 


1 Our previous report Are Kentucky’s Districts Closing the Achievement Gap? can be found at https://education.ky.gov/Com 
mOfEd/CDU/Documents/Closing%20the%20Achievement%20Gap.pdf 
2 Results for mathematics and other grade levels are available upon request. 


KEY FINDINGS 


DISTRICT ACHIEVEMENT GAP CONTEXTS VARY BEYOND CLOSING OR WIDENING AND CAN BE 
CATEGORIZED AS ONE OF FOUR TYPES OF GAP CHANGES: ACCELERATED CLOSURE, SLOWED CLOSURE, 
SLOWED WIDENING, AND ACCELERATED WIDENING. 


Figure 1 compares the distance from districts’ predicted third grade reading income gap in school year 2017-2018 to their actual 
gap change from 2016-2017 to 2017-2018. Amongst districts that saw closure in this achievement gap in 2017-2018 (those 
below the dashed line), two types of closure are evident. The first shows accelerated closure in quadrant II| (purple). These districts 
saw both a real decrease in their achievement gap, and their gap was smaller than predicted by the model. Thus they were able to 
decrease their third grade reading income gap by more than predicted, showing very promising action. It is of particular note that 
64 districts, or 38% of the 168 districts included in the analysis demonstrated this level of progress. Those in quadrant II (blue) also 
showed closure but at a slower pace. While their achievement gap decreased in magnitude, their 2017-2018 gap was larger than 


predicted. This means these districts decreased their gap by a smaller magnitude than predicted. 


This concept might be best illustrated by two example districts. Hickman County and McCreary County saw the same absolute 
change in their third grade reading income achievement gap between school years 2016-2017 and 2017-2018, a decrease of 3.7 


scale score points. However, Hickman’s 2017-2018 gap was smaller than predicted, while McCreary’s was larger than predicted, 
indicating that Hickman is seeing faster progress than McCreary. The reverse is true for districts that saw widening gaps in 2017- 
2018 (those above the dashed line). 


These four groups suggest that while a district may see their third grade reading income gap widen or narrow from year to year, 
variation exists within these two possibilities. This variation may have implications for how districts consider and address their gap. 
Those in the slowing groups may monitor their progress, while those in the accelerating groups might reallocate resources or adjust 
focus and supports to maintain necessary improvement. Such categories may also inform the work KDE conducts in its service to 
districts. For example, while accountability measures and designations highlight areas of need, the achievement gap group metrics 


may offer more nuanced views into district struggles. 


Figure 1. District Income Achievement Gap Closure Groups - Grade 3 Reading 
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Actual Gap Minus Predicted Gap, 2016-17 


THE DISTRICTS WITH THE MOST SEVERE THIRD GRADE READING INCOME GAPS SAW THE HIGHEST 
RATES OF ACCELERATED CLOSURE AND SLOWED CLOSURE COMBINED IN 2017-2018; HOWEVER, THOSE 
WITH RELATIVELY MODERATE OR SMALLER GAPS SHOWED LESS PROGRESS. 


To address the achievement gap across the state, it is helpful to consider which districts are most in need of gap closure. 
Presumably, these are the districts with the largest gap. As we would hope, this group saw the most progress in 2017-2018: 
almost 80% of the districts with 2016-17 third grade reading income gaps in the fourth quartile (between 10.8 and 30.3 scale 


score points) saw some type of gap closure (Figure 2). There is a very worrying 19 percent in the Accelerated Widening group, 


though, that may warrant further investigation. This group increased their gap by between 0.47 and 4.20 scale score points, 
and accounted for 12,273 third grade students in 2017-18, which was almost a quarter of all third grade students in the state. 
Promisingly, districts in the second through fourth quartiles saw rates of Accelerated Closure almost three times higher than those 
in the first quartile. However, each of quartiles one through three saw Accelerated Widening rates twice as high as the fourth 


quartile. 


It seems that progress is being made amongst the districts in most the dire circumstances, but many districts with large third grade 
reading income gap challenges are continuing to see regression. This may show the presence of a stronger push for districts with 
the largest gap, but less focus for those in the middle, and little focus for districts needing early gap intervention. This indicates a 


need for districts to address the achievement gap regardless of where they are on the continuum of severity. 


Figure 2. Income Achievement Gap Groups by Gap Quartile - Grade 3 Reading 


iN 
i] 


Achievement Gap Quartile, 2016-17 


Percent of Districts 


J Accelerated Widening [| Slowed Widening [| Slowed Closure I] Accelerated Closure 


MORE VARIATION EXISTS IN THE THIRD GRADE READING INCOME GAP CHANGES OF THE WIDENING 
DISTRICTS THAN IN THE CLOSURE DISTRICTS. 


In considering the changes districts saw in their achievement gap between the 2016-2017 and 2017-2018 school years, the 
magnitude of these changes relative to their initial gap in 2016-2017, or how big of a bite they took out of/added to the problem, 
provides further context. When looking at the absolute magnitude of districts’ third grade reading income gap changes, the 
Accelerated Widening group saw a wider distribution than the Accelerated Closure group (Figure 3). For example, 30% of the 
Accelerated Widening districts saw an increase in their achievement gap of more than 100%, whereas 3% of Accelerated Closure 
districts saw a decrease of similar magnitude. This pattern was also consistent for the Slowed Widening and Slowed Closure 


districts. 


This asymmetrical pace may indicate that third grade reading income gap closure is much more limited and slow across the 


Commonwealth than gap widening. Further, the variation amongst the widening districts could point to differences in context 


and challenge faced by each district. For example, gap closure for a district that saw a 10% increase in 2017-2018 may be quite 
different from a district that saw a 200% increase. In sum, the problem of the achievement gap manifests differently across 


districts, especially for those struggling the most. Or, to paraphrase Tolstoy, each struggling district is unhappy in its own way. 


In order to effectively address their third grade reading income gaps, districts should consider the magnitude of their gap changes 
and how they might rein in extreme widening or accelerate closure. In providing supports for districts, these findings indicate that 


KDE should consider how variation, particularly for struggling districts, might be taken into account. 


Figure 3. Districts' Absolute Percent Change in Income Achievment Gap - Grade 3 Reading 
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CONCLUSIONS 


To tackle the third grade reading income achievement gap, the above suggests it may be necessary to take into account variation 
across the population of districts in Kentucky. This variation manifests itself in the type of progress being made, the severity of the 
gap, and the magnitude of gap changes. Such information can be useful as we think about implications for school improvement. 
For instance, our findings suggest that continuous improvement efforts need to be considered differently depending on the 
severity and dynamism of a district’s achievement gap. While attention is typically given to large achievement gaps, districts with 


nascent issues also need to consider how they might prevent potential widening of their current gap. 


This issue is illustrated in schools’ accountability designations. If districts rely on whether their schools are identified as a 
Comprehensive School Improvement (CSI) or Targeted School Improvement (TSI) to identify problems, these broader measures 
could miss problems in their early stages or even larger challenges occurring in the district. For example, fifteen districts that had 


gaps in the first quartile and that were identified as having Accelerated Widening had no elementary schools identified as CSI 


or as TSI for economically disadvantaged. Similarly, five of the eight districts that had a 2016-27 gap in the fourth quartile 

and that were also identified as an Accelerated Widening district did not have any elementary schools identified as CSI or as 
TSI for economically disadvantaged students. Even within districts that have elementary schools identified as CSI or TSI for 
economically disadvantaged students, there is great variation in the pace of gap change, the severity of the gap, and the size of 
gap changes. Consequently, both districts and KDE should consider how variation is manifested within and among districts as 


well as the possible implications for work aimed at closing the gap. 


Even though the problem may seem daunting, there are many examples of districts showing large and sustained progress in 
closing their gaps. These successes are not limited to those districts with small gaps; there are a number of districts with larger 
challenges that are making great strides. Such cases provide us with ample opportunities to study best practices that will help 


refocus our work to support districts to ensure each and every student is able to pursue a successful future. 


SUMMARY OF METHODOLOGY 


For our study, we analyzed student-level data from the third grade K-PREP reading assessment for the 2016/17 and 2017/18 
schools years. These data were merged with files containing information on student demographics and program information. 
Next, we aggregated scale scores for each K-PREP tests by grade and district using information about the school and district that 
the student attended, and estimated within-year achievement gaps for districts’ racial/ethnic and socioeconomic groups. We used 
residual gain models to estimate districts’ progress on closing racial/ethnic and socioeconomic achievement gaps. Gain models 


included variables that controlled for student and district characteristics. 


Our study is limited by our decision to include district-level variables in the model. Their inclusion significantly decreased the 
precision of our estimates. For example, we found districts’ prior year racial and socioeconomic achievement gaps to be weakly 
correlated to their current year gap (r = .18 and r = .35, respectively). However, given this limitation, we believe an approach that 
takes multiple years of data into account to estimate the achievement gap to be an improvement over an approach that examines 


data from one school year. 


Program information: 
Office of the Commissioner 
300 Sower Blvd 

Frankfort, KY 40601 


